Automatic generation of software test cases from formal specifications

Software testing consumes a large percentage of total software development costs. Yet, it is still usually performed manually in a non rigorous fashion. While techniques, and limited automatic support, for the generation of test data from the actual code of the system under test have been well researched, test cases generation from a high level speci cation of the intended behaviour of the system being developed has hardly been addressed. In this thesis we present a rationale for using tests derived from high level formal speci cations and then set to nd an e cient technique for the generation of adequate test sets from speci cations written in our study language, VDM-SL. In this work, we formalise the traditional high level partitioning technique used in a previously researched test cases generator prototype, and extend it to take the semantics of VDM-SL fully into account. We then discuss, and illustrate, the shortcomings of the technique as used, which results in too few tests being generated and potentially large sections of a speci cation not employed by the test generation process. Another strand of research, based on test generation from Z predicates, is examined and extended using our formalism to include quanti ed expressions and other, more complex, constructs of formal speci cation languages. We then show that this more re ned technique complements the previous work and that their combination should allow the generation of adequate test sets from formal speci cations. Using our formalism, we illustrate that to synthesise pragmatically the two techniques, one must nd heuristics for the detection of redundant test cases. We present our central heuristic, justi ed using probabilities, based on the independence of some divisions of the input domain of the system under test, which allows the contraction of test sets without impairing their likelihood of revealing an error in the system under test. Finally, we propose a technique for the e cient generation of adequate test cases. Acknowledgements I wish to thank Dr. Ivor Spence, my supervisor, for his help and patience during the course of this work. I would also like to thank Mr. John Clark and Prof. John McDermid of the university of York for allowing me to pursue my work while being employed on one of their projects. This work was partially funded in its rst year by the SERC (grant number 92313295). It was generously funded by a EU Human Capital and Mobility grant (number ERBCHB CT93 0328) for two years. Prof. Maurice Clint and the department of computer science at Queen's at large, must be thanked for their e orts in obtaining this latest grant. Finally, I am indebted to my wife, Tracey, and my son Mikael for putting up with me during di cult times and encouraging me all the way. i To the memory of my parents. ii